Skip to main content

TAI - Assess Bias

TAI - Assess Bias​

ACCESS THE CUSTOM STEP FOLDER

TAI - Assess Bias

Description​

This custom step allows the user to evaluate predictive model bias across sensitive groups using the SAS procedure PROC ASSESSBIAS. The step takes a scored input table (containing a predicted probability column, (P_[ColumnName]) produced by a previous statistical/modeling step) and produces a series of tables that help the user understand fairness in their model:

  • Group Metrics — including number of observations, average squared error, and absolute error per sensitive group.
  • Max Difference Between Sensitive Groups — depicting the "worst case" disparity across groups.
  • Bias Metrics — derived from the group metrics, capturing differences in average prediction, misclassification rate, and related measures.

The closer a bias metric value is to 0, the more similar the treatment was across sensitive groups.

More information about PROC ASSESSBIAS: https://go.documentation.sas.com/doc/en/pgmsascdc/default/casactml/casactml_assessbias_toc.htm

User Interface​

Configuration tab​

  • Input table — Select the scored table that contains the predicted probability column, the target variable, and the sensitive variable.
  • Predicted Probability Variable — Select one or more probability columns (typically P_[TargetEvent]) generated by an upstream modeling step.
  • Sensitive Variable — Select exactly one column representing the sensitive grouping (e.g., a PII or demographic variable) on which bias will be assessed.
  • Target section
    • Target Variable — Select the actual target/outcome column.
    • Target Level — Choose Interval or Nominal.
    • Target Event (visible when Target Level = Nominal) — Select the event level of interest from the target variable.
    • FitStat subsection
      • P Variables — Optional column used in the fitstat statement.
      • P Event — Optional event for the FitStat probability variable, referenced from the target variable.
      • Delimiter (visible when P Event is set) — Choose the delimiter used by the FitStat statement: Space, ;, *, ., or ,.
      • TAI - Assess Bias Configuration Screen

Options tab​

  • Cutoff Ratio — Classification cutoff for predicted probabilities (range 0–1, default 0.5).
  • Number Of Bins — Number of bins used for binned bias metrics (min 2, default 20).
  • Number Of Cuts — Number of cutpoints used for cut-based bias metrics (range 2–100, default 100).
  • Selection Depth — Selection depth percentage for depth-based metrics (range 1–100, default 10).
  • Frequency — Optional frequency column.
  • Weight — Optional observation weight column.
  • TAI - Assess Bias Options Screen

Requirements​

Usage​

  • Required inputs: Input table, Predicted Probability Variable, Sensitive Variable, and Target Variable. If any of these are missing, the step will not execute PROC ASSESSBIAS; instead, an error messege saying PROC ASSESSBIAS will not run because one or more required variables are missing.
  • For Nominal targets, one of two fitstats will occur:
    • If P Variables and P Event are provided, the step will perform fitstat pVar=<pVars> / pEvent="<pEvent>" Delimiter="<delimiter>", where the delimiter is declared via the setDelimiter macro.
    • setDelimieter will default to a space delimiter but can also be changed to ; (Semicolon), * (Astrisk), . (Period), or , (Comma).
    • If only P Variables is provided, the step emits a base fitstat pVar=<pVars> with no pEvent or delimiter.
    • If P Variables are not provided, no fitstat statement is performed.
  • The optional Weight and Frequency columns are performed only when populated.

Change Log​

  • Version 1.0.15 (26JUN2026)

    • Added Cleanup for global and helper macros in source to improve memory and security
    • Modified UI so that all options have action verbs (Input table -> Select an input table)
    • Fixed mis-formating in README so that changes are indented under the appropriate version

  • Version 1.0.14 (25JUN2026)

    • Begining of updating change log
    • Re-organized SAS code to have macros in macro section and execution code in execution section
    • Modified UI to follow (UI Guidelines)

  • Version 1.0.0 (19JUN2026)

    • Initial version